Explaining Queries over Web Tables to Non-Experts

Designing a reliable natural language (NL) interface for querying tables has been a longtime goal of researchers in both the data management and natural language processing (NLP) communities. Such an interface receives as input an NL question, translates it into a formal query, executes the query and returns the results. Errors in the translation process are not uncommon, and users typically struggle to understand whether their query has been mapped correctly. We address this problem by explaining the obtained formal queries to non-expert users. Two methods for query explanations are presented: the first translates queries into NL, while the second method provides a graphic representation of the query cell-based provenance (in its execution on a given table). Our solution augments a state-of-the-art NL interface over web tables, enhancing it in both its training and deployment phase. Experiments, including a user study conducted on Amazon Mechanical Turk, show our solution to improve both the correctness and reliability of an NL interface.Comment: Short paper version to appear in ICDE 201

Answering Questions by Meta-Reasoning over Multiple Chains of Thought

Modern systems for multi-hop question answering (QA) typically break questions into a sequence of reasoning steps, termed chain-of-thought (CoT), before arriving at a final answer. Often, multiple chains are sampled and aggregated through a voting mechanism over the final answers, but the intermediate steps themselves are discarded. While such approaches improve performance, they do not consider the relations between intermediate steps across chains and do not provide a unified explanation for the predicted answer. We introduce Multi-Chain Reasoning (MCR), an approach which prompts large language models to meta-reason over multiple chains of thought, rather than aggregating their answers. MCR examines different reasoning chains, mixes information between them and selects the most relevant facts in generating an explanation and predicting the answer. MCR outperforms strong baselines on 7 multi-hop QA datasets. Moreover, our analysis reveals that MCR explanations exhibit high quality, enabling humans to verify its answers

MRKAd5 HIV-1 Gag/Pol/Nef Vaccine-Induced T-Cell Responses Inadequately Predict Distance of Breakthrough HIV-1 Sequences to the Vaccine or Viral Load

Background: The sieve analysis for the Step trial found evidence that breakthrough HIV-1 sequences for MRKAd5/HIV-1 Gag/Pol/Nef vaccine recipients were more divergent from the vaccine insert than placebo sequences in regions with predicted epitopes. We linked the viral sequence data with immune response and acute viral load data to explore mechanisms for and consequences of the observed sieve effect. Methods: Ninety-one male participants (37 placebo and 54 vaccine recipients) were included; viral sequences were obtained at the time of HIV-1 diagnosis. T-cell responses were measured 4 weeks post-second vaccination and at the first or second week post-diagnosis. Acute viral load was obtained at RNA-positive and antibody-negative visits. Findings: Vaccine recipients had a greater magnitude of post-infection CD8+ T cell response than placebo recipients (median 1.68% vs 1.18%; p = 0.04) and greater breadth of post-infection response (median 4.5 vs 2; p = 0.06). Viral sequences for vaccine recipients were marginally more divergent from the insert than placebo sequences in regions of Nef targeted by pre-infection immune responses (p = 0.04; Pol p = 0.13; Gag p = 0.89). Magnitude and breadth of pre-infection responses did not correlate with distance of the viral sequence to the insert (p. 0.50). Acute log viral load trended lower in vaccine versus placebo recipients (estimated mean 4.7 vs 5.1) but the difference was not significant (p = 0.27). Neither was acute viral load associated with distance of the viral sequence to the insert (p>0.30). Interpretation: Despite evidence of anamnestic responses, the sieve effect was not well explained by available measures of T-cell immunogenicity. Sequence divergence from the vaccine was not significantly associated with acute viral load. While point estimates suggested weak vaccine suppression of viral load, the result was not significant and more viral load data would be needed to detect suppression.National Institute of Allergy and Infectious Diseases [R37AI054165-08, UM1AI068635, UM1AI068618]National Institute of Allergy and Infectious Disease

QAMPARI: : An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs

Existing benchmarks for open-domain question answering (ODQA) typically focus on questions whose answers can be extracted from a single paragraph. By contrast, many natural questions, such as "What players were drafted by the Brooklyn Nets?" have a list of answers. Answering such questions requires retrieving and reading from many passages, in a large corpus. We introduce QAMPARI, an ODQA benchmark, where question answers are lists of entities, spread across many paragraphs. We created QAMPARI by (a) generating questions with multiple answers from Wikipedia's knowledge graph and tables, (b) automatically pairing answers with supporting evidence in Wikipedia paragraphs, and (c) manually paraphrasing questions and validating each answer. We train ODQA models from the retrieve-and-read family and find that QAMPARI is challenging in terms of both passage retrieval and answer generation, reaching an F1 score of 26.6 at best. Our results highlight the need for developing ODQA models that handle a broad range of question types, including single and multi-answer questions

Post-Infection Magnitude of CD8+ T-Cell Response.

<p>Magnitude of the post-infection CD8+ T-cell response measured by ICS, as quantified by the percentage of CD8+ T-cells producing IFN or IL-2 when stimulated with the vaccine-insert-matched peptide pools (Gag, Pol, and Nef) and other non-vaccine-insert peptide pools, for vaccine and placebo groups. Positive responses are indicated using closed red circles and negative responses using open blue circles. The p-values refer to tests comparing response magnitudes between the vaccine and placebo positive responders.</p

Post-Infection Breadth of T-Cell Response.

<p>Breath of the post-infection T-cell response as measured by IFNγ ELISpot, as quantified by the number of reactive 15-mers, for the vaccine (grey) and placebo (black) groups. The distribution of breadth is shown for all proteins in aggregate; for Gag, Pol, and Nef combined; for other non-insert proteins; and for Gag, Pol, and Nef individually. The p-values refer to tests comparing breadth between vaccine and placebo groups.</p

Acute Log₁₀ Viral Load.

<p>The distribution of acute log₁₀ viral load values in vaccine and placebo groups. Solid lines correspond to observed means and dashed lines correspond to means estimated using the multiple imputation approach.</p